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PRECISION  OF  SIMULTANEOUS  MEASUREMENT  PROCEDURES 

W.  A.  Thompson,  Jr. 

National  Bureau  of  Standards 
Washington,  D.  C. 

1.  ABSTRAC T.  We  consider  the  problem  of  measurement  under  the 
following  conditions:  The  process  of  gathering  the  data  is  such  that  on  any- 
given  item  only  one  opportunity  for  measurement  occurs,  but  it  can  be 
observed  simultaneously  by  several  instruments,  The  items  to  be  meas- 
ured are  variable  so  that  one  cannot  obtain  replicate  observations  with  the 
same  instrument  which  would  show  directly  the  variance  of  the  instrument 
■readings.  Procedures  are  discussed  for  estimating  the  precisions  of  the 
instruments  and  the  variability  of  the  items  being  measured. 

An  example  due  to  Simon  and  Grubbs  is  helpful  in  fixing  ideas.  The 
burning  times  of  thirty  similar  fuzes  are  determined  by  several  different 
observers.  We  limit  our  discussion  to  the  data  taken  by  observers  A and 
C;  hence  there  are  two  determinations  of  the  burning  times  of  thirty  differ- 
ent fuzes  or  sixty  observations  in  all.  If  each  of  the  fuzes  had  the  same  run- 
ning time  (which  is  the  manufacturer's  goal)  and  if  both  of  the  observers 
were  absolutely  accurate,  then  all  sixty  observations  would  be  equal.  How- 
ever, considerable  inequality  in  such  data  always  occurs  due  to  variation  in 
the  manufacturing  process  and  inaccuracy  of  the  observations.  It  then 
becomes  desirable  to  use  the  sixty  observations  to  answer  as  many  ques- 
tions as  possible  about  measurement  bias  and  precision,  mean  fuze  running 
time,  and  variability  of  burning  times  about  their  mean. 

2.  THE  MODEL.  With  the  verbal  description  of  the  previous  section 

in  mind,  consider  the  following  mathematical  formulation.  Let  x^ x 

denote  the  true  values  of  the  items  to  be  measured.  Assume  that  x.,  . . . , x„„ 

constitute  a random  sample  of  size  N selected  from  a population  having 
mean  fA  and  variance  cr  2 . Each  of  the  items  in  this  sample  is  then 

measured  by  p instruments,  y. . is  the  measurement  of  the  i**1  item 
(i=l,  . . . ,N)  according  to  the  jth  instrument  (j  = 1,  . . . ,p  ).  The  consequence 
of  this  measurement  is  that  an  instrumentation  error  e..,  chosen  at  random 

th  1J 

from  the  j instrument's  population  of  errors,  is  added  to  the  true  value 

of  the  i*“  item: 


= x.  + e, .. 
i ij 


(2.1) 
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The  instrumentation  errors  are  taken  to  be  uncorrelated  among 
themselves  and  also  uncorrelated  with  the  items  selected  for  measure- 
ment. The  mean  and  variance  of  e..  are/j  . and  cr  2 respectively:  A 

th  13  J J 

may  be  called  the  bias  of  the  j-  instrument. 

Denoting  the  vector  (y^,  . . . , y ) by  we  may  think  of  Y^,  . . . , Y^ 

as  constituting  a sample  of  size  N = n+1  from  a p-variate  distribution 
with  mean  vector  (H  +^,  . . . , + ^ ) and  dispersion  matrix 


2 , 

(T  + (T 


(2.2) 


2 , 2 
1 r + ff2 


2 ' 2 

or  + <r 

P 


Notice,  in  passing,  that  if  all  instrument  variances  are  equal,  then 
the  model  becomes  a completely  random  one-way  layout  and  may  be 
analysed  by  the  methods  which  appear,  for  example,  in  Scheffe  |5|  . 

A paragraph  on  notation  will  perhaps  be  helpful.  £ . will  be  used 
as  a more  succinct  notation  forp-+^j,  j = 1,  . . . , p.  will  frequently 

write  3T,  = (cr  ../)  when  we  mean  that  cr  / is  the  element  in  the  j**1  row 

th  JJ  -1  'S' 

and  y column  of  In  the  same  spirit  J’  = («r  JJ  ),  A = (a„/)  and 

S = (s.,/)  will  be  common  notations.  Here 
JJ 


(2.3) 


a 1 

a.t/  = (y-.  - y-  •)  (y../  - y- ./) 

jr  1=1  ij  j ij  j 


and  s / is  the  usual  unbiased  estimate  of  cr  ../,  i.e.  s = a,./  / n. 

JJ  JJ  JJ  JJ 

3.  POINT  ESTIMATES.  In  ^2j  , Grubbs  recommends  certain 
estimates  of  item  and  instrument  variance.  For  p = 2 instruments, 
these  estimates  are 
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(3.1) 


S12’  T1 


11 


S12andtr2' 


S22  ' S12’ 


where  /—is  to  be  read  “is  estimated  by".  For  p 7_3,  Grubbs  recommends 
2 2 


pCp-1) 


3<3/ 


s../  , 
JJ 


(3.2) 


+ (P-1)  (p-2) 


2 


JJ' 


with  an  analogeous  estimate  of  the  other  instrument  variances.  Gaylor  [\] 
shows  that  Grubbs*  estimate  of  or  is  equivalent  to  a familiar  variance 
component  estimate.  These  estimates  are  reasonable  in  that  they  have 
the  correct  dimensionality,  are  unbiased,  and  have  appropriate  symmetry 
properties  when  the  instrument  labels  are  interchanged.  Further,  if  the 
underlying  distribution  is  normal,  then  Grubbs'  estimates  are  simple 
functions  of  the  sufficient  statistics  and  in  the  case  p=2  they  have  a max- 
imum likelihood  property.  However,  as  Grubbs  has  verbally  pointed  out 
his  estimates  are  unreasonable  in  that  they  frequently  assume  negative 
values  even  though  the  parameters  themselves  must  be  non-negative  by 
their  very  definition. 


For  p=2  this  objectional  characteristic  has  been  eliminated  in  Eh 
here  the  altered  estimates  of  table  1 have  been  proposed.  The  top  line 
of  this  table  yields  the  estimates  (3.  1)  under  conditions  where  they  are 
non-negative.  The  remaining  entries  show  how  Grubbs'  estimates  can  be 
modified  when  negativity  would  result  from  using  (3.  1).  These  modified 
estimates  have  been  derived,  under  normality  assumptions,  from  a prin- 
ciple of  restricted  maximum  likelihood  which  is  fairly  well  accepted  in 
other  brances  of  statistical  practice.  A tilda  placed  over  a parameter  in- 
dicates its  restricted  maximum  likelihood  estimate. 
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Table  1.  Non-negative  estimates  in  the  two  instrument-case. 


Conditions 

<T 

2 

ffl 

2 

* 2 

S11>S12 

S12 

S11  " S12 

S22  " S12 

s22>e12>° 

S22  — S12>S11 

S11 

0 

sn  +s22  • 2s12 

Sll>  S12  >S22 

S22 

S11  + 622  ' 2s12 

0 

S12<° 

0 

S11 

s22 
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4.  RELATIVE  PRECISION.  It  is  clear  that  if  the  instrumentation 
of  an  experiment  is  to  be  effective  then  the  instruments  must  be  precise 
relative  to  the  variability  of  the  quantity  being  measured.  A frequently 
quoted  rule  of  thumb  is  that  the  instrument  precision  should  be  an  order 
of  magnitude  greater  than  that  of  the  item  being  measured.  Such  a 
statement  has  no  firm  meaning  unless  a measure  of  instrument  precision 
and  a measure  of  item  variability  have  been  agreed  upon.  Here  we  adopt 
= cr  / <j  ^ as  a measure  of  the  relative  precision  of  the  first  instrument. 

Then,  for  example,  the  above  mentioned  rule  of  thumb  would  become 

Ai>io. 

In  the  two -instrument  case,  assuming  normality,  we  may,use  a result 
of  Roy  and  Bose  ^3j  to  make  inferential  statements  of  a statistical  nature 
concerning  the  parameter  ^ . In  our  terminology  their  result  states  that 


(4.1) 


'n-l" 

‘ii  [m 


I I 2 

has  the  t- distribution  with  n - 1 d.  f.  where  A|  = a^a  ” ai2’ 

Noting  that  or  ^/<r  ^ ^ = (l  we  maY  verify  that  the  quantity  (4.  1) 

is  less  than  t^,  if  and  only  if  * 


(4.2) 


ai2'V  H. 

a 2 f*  n-l 

~ [a  f/a- 

an  - a + t -1 — *- 
11  12  (X  n-l 


Hence  if  t^  is  the  upper  0(  percentage  point  of  the  t-  distribution  with 

n-l  d.  f.  then  the  square  root  of  the  right  hand  side  of  (4.  2)  provides  a 
lower  confidence  bound  for  A,  » the  confidence  coefficient  being  1-^. 

The  inequality  (4.  2)  can  also  Tie  used  for  the  purpose  of  hypothesis  testing 
For  example,  we  may  rejectA  10  at  the  significance  level  o(  if  (4.  2)  is 
violated  with  ^ 2 


5.  A SIMULTANEOUS  CONFIDENCE  REGION.  For  some  purposes 


it  may  not  be  enough  to  consider  relative  precision;  we  may  be  interested  in 
the  actual  non-relative  precisions  and  the  item  variability.  Estimation  of 
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the  parameters  tr  , or  , . . . , o-  was  discussed  in  section  3;  but  how 

reliable  are  estimates  ? This  question  is  dealth  with  in  , again  under 
assumptions  of  normality. 


In  the  two -instrument  case,  the  probability  is  at  least  1 - 2 a.  that 
the  following  three  relations  hold  simultaneously 


(5.1) 


i =1,2. 


Here  K and  M are  to  be  found  in  Table  2 under  the  desired  value  of  2°< . 


Tabic  2.  The  table  gives  values  of  K and  M which  181 

yield  1 - 2a  confidence  regions  when  used  in  conjunction 
with  the  relations  (5.1). 


2a  .01 .05 


n 

K 

LI 

K 

k 

3 

99.78 

99.72 

19.79 

19.71 

4 

12.38 

12.33 

4.146 

4.077 

5 

3.980 

3.931 

1.726 

1.665 

6 

1.903 

1.353 

.9636 

.9083 

7 

1.120 

1.078 

.6290 

.5786 

8 

0.7459 

.7076 

.4516 

.4052 

8 

0.5389 

.5031 

.3453 

.3022 

10 

0.4120 

.3782 

.2761 

.2357 

11 

0.3282 

.2963 

.2230 

.1901 

12 

0.2698 

.2395 

.1932 

.1573 

13 

0.2272 

.1983 

.1663 

.1328 

14 

0.1951 

.1675 

.1464 

.1140 

15 

0.1702 

.1438 

.1301 

.09925 

16 

0.1505 

.1251 

.1169 

.03738 

17 

0.1344 

.1100 

.1060 

.07767 

18 

0.1213 

.09772 

.09632 

.06962 

19 

0.1103 

.08752 

.08904 

.06237 

20 

0.1009 

.07896 

.08237 

.05713 

22 

.08610 

.06546 

.07152 

.04795 

24 

.07484 

.05538 

.06311 

.04098 

26 

.06605 

.04763 

.05641 

.03554 

28 

.05901 

.04152 

.05096 

.03121 

30 

.05328 

.03660 

.04644 

.02768 

35 

.04272 

.02778 

.03796 

.02127 

40 

.03556 

.02200 

.03205 

.01700 

45 

.03040 

.01797 

.02771 

.01398 

50 

.02652 

,01503 

.02440 

.01176 

60 

.02109 

,01110 

.01967 

.00875 

70 

.01748 

.00362 

.01646 

.00684 

80 

.01492 

.00694 

.01415 

.00553 

SO 

.01300 

.00575 

.01241 

.00460 

100 

.01152 

.00486 

.01104 

.00390 

Page  Blank 
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For  more  than  two  instruments  a similar  result  is  valid.  We  have 
the  following  relations  with  probability  at  least  1 - 2 A 


(5.2)  max  [ a. K - M (a.,  a 2 ] 

j f j/  1 JJ  ]]  J 

<Tc r < min  fa  . K + M(a. . a.  . ) 2 1 

~ ”j  / j/L  jj  JJ  33  ■*  J 

ix 

J / 1 


mi?x  , [(au  - a!j>  K - M[an  + ajj  -2aij)  ] 


2<  min  ^ [ (au  - at.)K  + M [«u  + a..  » Za^)]  2J, 


plus  p - 1 similar  inequalities  involving  cr  , . . . , <r  . Unfortunately, 

2 p 

for  p in  excess  of  two,  tables  of  K and  M are  unavailable.  The  only 
result  which  is  currently  ready  for  use  is  an  approximation  valid  for 
large  n:  Choose  X to  satisfy  P(j£  p+2)  = 1 “ 2tfT,  write  K = M = l/2i 

and  substitute  this  common  value  in  (5.  2).  I feel  obliged  to  point  out 
that  for  p = 2,  the  only  case  wiiere  exact  values  are  available,  this 
approximation  is  rather  poor. 


6.  NUMERICAL  EXAMPLE.  Returning  to  the  fuze  burning  time 
data,  we  may  identify  observer  A as  the  first  instrument  and  observer 
C as  the  second.  From  Table  I of  Grubbs’  paper  [2]  we  obtain 
a^  =1.3671,  a^  =1.3227,  a^  =1.3320  and  n = -29.  F rom  the  third 

rv  ^ 

row  entry  of  our  table  1,  we  estimate  cr  = . 21,  cr  = .03  and  c r ^ = 0. 

By  the  method  of  section  4 we  obtain,  for  example,  that  the  relative 
precision  of  observer  C exceeds  5.  1 with  a confidence  of  95%. 
Alternatively,  from  a hypothesis  testing  point  of  view  we  would  reject 
the  rule  of  thumb  requirement,  A _ 3>  10,  at  the  5%  level.  The  relations 

(5.  1)  and  table  2 yield  the  following  95%  simultaneous  confidence  region: 

. 1 6<  cr  ^ . 32,  0 ^~cr  < . 09  and  0<^cr  ' <C  . 07.  In  calculating  these 

simultaneous  confidence  intervals  we  have  replaced  all  negative  lower 
bounds  by  zero.  Notice  that  the  confidence  intervals  bracket  their 
respective  estimates  and  hence,  in  the  confidence  region  sense,  indicate 
the  uncertainty  of  these  estimates. 
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